Medical image processing system

ABSTRACT

Provided is a medical image processing system capable of maintaining or improving the resolution of an image obtained by using narrow-band light of a short wavelength while maintaining image quality of an image obtained by using broadband light such as white light. A light source unit emits specific narrow-band light of a short wavelength. An image sensor includes a first pixel group including a B pixel and a second pixel group including a G pixel and a W pixel. The B pixel has a higher sensitivity to the specific narrow-band light than the G pixel. The G pixel has a sensitivity to light in a green band and the specific narrow-band light. The W pixel has a sensitivity to broadband illumination light including the light in the green band and the specific narrow-band light.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of PCT International Application No. PCT/JP2019/015760 filed on Apr. 11, 2019, which claims priority under 35 U.S.C § 119(a) to Japanese Patent Application No. 2018-082157 filed on Apr. 23, 2018. Each of the above application(s) is hereby expressly incorporated by reference, in its entirety, into the present application.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to a medical image processing system that illuminates an observation target with narrow-band light of a short wavelength and images the observation target with a color image sensor.

2. Description of the Related Art

In the medical field, an endoscope system comprising a light source device, an endoscope, and a processor device is widely used. In the endoscope system, an observation target is irradiated with illumination light from the endoscope, and an image of the observation target is displayed on the monitor on the basis of an RGB image signal obtained by imaging the observation target illuminated with the illumination light with an image sensor of the endoscope.

As an imaging element, in addition to an RGB image sensor having a Bayer array in which the ratio of the number of pixels of G pixels, B pixels, and R pixels is 2:1:1, an RGBW image sensor provided with W pixels having a sensitivity to broadband light such as white light in addition to the B pixels, the G pixels, and the R pixels is being used. In the RGBW image sensor, similar to the RGB image sensor having a Bayer array, demosaicing processing is performed to generate RGB image signals of three colors at each pixel position (for example, JP5141757B and JP2011-055038A).

SUMMARY OF THE INVENTION

In endoscopic diagnosis in recent years, in order to observe specific structures such as surface blood vessels and red blood cells, narrow-band light of a short wavelength having a center wavelength of 410 nm to 450 nm is being used for illumination of an observation target. In a case where such an observation target illuminated with narrow-band light of a short wavelength is imaged by a color image sensor such as an RGB image sensor or an RGBW image sensor, the number of pixels of B pixels having a sensitivity to narrow-band light of a short wavelength is smaller than the number of pixels of other pixels having almost no sensitivity to narrow-band light of a short wavelength. Accordingly, an image obtained by the color image sensor has a reduced resolution after the demosaicing processing. Therefore, it is required to maintain or improve the resolution of an image obtained by using narrow-band light of a short wavelength while maintaining image quality of an image obtained by using broadband light such as white light.

An object of the invention is to provide a medical image processing system that can maintain or improve the resolution of an image obtained by using narrow-band light of a short wavelength while maintaining image quality of an image obtained by using broadband light such as white light.

A medical image processing system according to an aspect of the invention comprises a light source unit that emits specific narrow-band light of a short wavelength, and an image sensor that images an observation target illuminated with the specific narrow-band light, the image sensor including a first pixel group including a first pixel and a second pixel group including at least a second pixel and a third pixel. The first pixel has a higher sensitivity to the specific narrow-band light than the second pixel, the second pixel has a sensitivity to first long-wavelength light of a longer wavelength than the specific narrow-band light and the specific narrow-band light, and the third pixel has a sensitivity to broadband illumination light including the specific narrow-band light and the first long-wavelength light.

It is preferable that, in the sensitivity of the second pixel, a short-wavelength sensitivity to short-wavelength light including the specific narrow-band light is 10% or more of a maximum sensitivity of the second pixel, or 10% or more of the short-wavelength sensitivity of the first pixel. It is preferable that, in the sensitivity of the second pixel, a short-wavelength sensitivity to short-wavelength light including the specific narrow-band light is 35% or less of a maximum sensitivity of the second pixel, or 35% or less of the short-wavelength sensitivity of the first pixel. It is preferable that the sensitivity of the third pixel is 30% or more and 50% or less with respect to a maximum sensitivity of the first pixel or the second pixel.

It is preferable that a pixel value of the first pixel at a pixel position of the second pixel is calculated by demosaicing processing based on a correlation with a pixel value of the third pixel. It is preferable that the demosaicing processing includes acquiring an image blur reduction processed signal in which image blur reduction processing is performed on an image signal output from the image sensor, acquiring a first smooth-filtered component by multiplying the image signal by a first smoothing filter having a filter coefficient at a position corresponding to the third pixel, and acquiring a second smooth-filtered component by multiplying the image signal by a second smoothing filter that is applied to each specific pixel area in which the second pixel is located at a specific position, the second smoothing filter having a filter coefficient at a position corresponding to the first pixel in the specific pixel area, and the pixel value of the first pixel at the pixel position of the second pixel is calculated on the basis of the image blur reduction processed signal, the first smooth-filtered component, and the second smooth-filtered component.

It is preferable that the second pixel group includes a fourth pixel, and the fourth pixel has a sensitivity to second long-wavelength light of a longer wavelength than the first long-wavelength light and the specific narrow-band light. It is preferable that, in the sensitivity of the fourth pixel, a short-wavelength sensitivity to short-wavelength light including the specific narrow-band light is 10% or more of a maximum sensitivity of the fourth pixel, or 10% or more of the short-wavelength sensitivity of the first pixel. It is preferable that, in the sensitivity of the fourth pixel, a short-wavelength sensitivity to short-wavelength light including the specific narrow-band light is 35% or less of a maximum sensitivity of the fourth pixel, or 35% or less of the short-wavelength sensitivity of the first pixel.

It is preferable that a pixel value of the first pixel at a pixel position of the fourth pixel is calculated by demosaicing processing based on a correlation with a pixel value of the third pixel. It is preferable that the demosaicing processing includes acquiring an image blur reduction processed signal in which image blur reduction processing is performed on an image signal output from the image sensor, acquiring a first smooth-filtered component by multiplying the image signal by a first smoothing filter having a filter coefficient at a position corresponding to the third pixel, and acquiring a third smooth-filtered component by multiplying the image signal by a third smoothing filter that is applied to a specific pixel area in which the fourth pixel is located at a specific position, the third smoothing filter having a filter coefficient at a position corresponding to the first pixel in the specific pixel area, and the pixel value of the first pixel at the pixel position of the fourth pixel is calculated on the basis of the image blur reduction processed signal, the first smooth-filtered component, and the third smooth-filtered component.

It is preferable that the number of pixels of the second pixel group is greater than the number of pixels of the first pixel group. It is preferable that a center wavelength of the specific narrow-band light is included in a range of 400 nm to 450 nm, and a half-width of the specific narrow-band light is 40 nm or less.

According to the invention, it is possible to maintain or improve the resolution of an image obtained by using narrow-band light of a short wavelength while maintaining image quality of an image obtained by using broadband light such as white light.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an external view of an endoscope system.

FIG. 2 is a block diagram showing a function of the endoscope system.

FIG. 3 is a graph showing spectra of violet light V, blue light B, green light G, and red light R.

FIG. 4 is a graph showing a spectrum of specific narrow-band light of a short wavelength.

FIG. 5 is an explanatory diagram showing R pixels (R), G pixels (G), B pixels (B), and W pixels (W) provided in an image sensor.

FIG. 6 is a graph showing spectral transmittances of a B filter, a G filter, and an R filter included in an image sensor of the present embodiment.

FIG. 7 is an explanatory diagram showing received light intensities of a B pixel and a G pixel in a case where an esophagus is illuminated with specific narrow-band light of a short wavelength.

FIG. 8 is an explanatory diagram showing received light intensities of a B pixel and an R pixel in a case where an esophagus is illuminated with specific narrow-band light of a short wavelength.

FIG. 9 is an explanatory diagram showing received light intensities of a W pixel and a G pixel in a case where an observation target is illuminated with normal light.

FIG. 10 is an explanatory diagram showing received light intensities of a W pixel and an R pixel in a case where an esophagus is illuminated with normal light.

FIG. 11 is an explanatory diagram showing a G pixel located at a specific position SP in an image signal.

FIG. 12 is an explanatory diagram showing a method of calculating a low frequency component mW.

FIG. 13 is an explanatory diagram showing a method of calculating a low frequency component mBx.

FIG. 14 is an explanatory diagram showing a method of calculating a low frequency component mBy.

FIG. 15 is a graph showing spectral transmittances of a B filter, a G filter, and an R filter included in a normal RGB image sensor.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

As shown in FIG. 1, an endoscope system 10 has an endoscope 12, a light source device 14, a processor device 16, a monitor 18, and a user interface 19. The endoscope 12 is optically connected to the light source device 14 and electrically connected to the processor device 16. The endoscope 12 has an insertion part 12 a to be inserted into a subject, an operation part 12 b provided in a proximal end portion of the insertion part 12 a, and a bendable portion 12 c and a distal end portion 12 d that are provided on the distal end side of the insertion part 12 a. The bendable portion 12 c is bent by operating an angle knob 12 e of the operation part 12 b. With the bending operation, the distal end portion 12 d is directed in a desired direction. The user interface 19 includes a mouse or the like in addition to the illustrated keyboard.

In addition to the angle knob 12 e, a mode switching SW 13 a and a still image acquisition instruction part 13 b are provided in the operation part 12 b. The mode switching SW 13 a is used to switch among a normal observation mode, a special observation mode, and a short wavelength observation mode. The normal observation mode is a mode in which an observation target is illuminated with normal light such as white light (see FIGS. 9 and 10) and a normal image is displayed on the monitor 18. The special observation mode is a mode in which an observation target is illuminated with special light such as blue narrow-band light, and a special observation image in which a structure such as a blood vessel at a specific depth is emphasized is displayed on the monitor 18. The short wavelength observation mode is a mode in which a short wavelength observation image showing a structure that can be observed with specific narrow-band light of a short wavelength (corresponding to violet light V described later) is displayed on the monitor 18 by using the specific narrow-band light of the short wavelength for illumination of an observation target. As the mode switching unit for switching the mode, a foot switch may be used in addition to the mode switching SW 13 a.

The processor device 16 is electrically connected to the monitor 18 and the user interface 19. The monitor 18 outputs and displays image information and the like. The user interface 19 functions as a user interface (UI) that receives input operations such as function settings. An external recording unit (not shown) for recording image information and the like may be connected to the processor device 16.

As shown in FIG. 2, the light source device 14 has a light source unit 20, a light source control unit 21, and an optical path coupling unit 23. The light source unit 20 has a violet light emitting diode (V-LED) 20 a, a blue light emitting diode (B-LED) 20 b, a green light emitting diode (G-LED) 20 c, and a red light emitting diode (R-LED) 20 d. The light source control unit 21 controls driving of the LEDs 20 a to 20 d. The optical path coupling unit 23 couples optical paths of light of four colors emitted from the LEDs 20 a to 20 d of four colors. The light coupled by the optical path coupling unit 23 is emitted into the subject through a light guide 41 and an illumination lens 45 which are inserted into the insertion part 12 a. A laser diode (LD) may be used instead of the LED.

As shown in FIG. 3, the V-LED 20 a generates violet light V having a center wavelength of 400 nm to 450 nm (for example, 405 nm) and a half-width of 40 nm or less. The B-LED 20 b generates blue light B having a center wavelength of 450 nm to 500 nm (for example, 460 nm) and a half-width of 40 nm or less. The G-LED 20 c generates green light G having a wavelength range of 480 nm to 600 nm. The R-LED 20 d generates red light R having a center wavelength of 620 nm to 630 nm and a wavelength range of 600 nm to 650 nm.

The light source control unit 21 performs control to turn on the V-LED 20 a, the B-LED 20 b, the G-LED 20 c, and the R-LED 20 d in the normal observation mode and the special observation mode. In the normal observation mode, the light source control unit 21 controls each of the LEDs 20 a to 20 d so as to emit normal light in which a light intensity ratio among the violet light V, the blue light B, the green light G, and the red light R is Vc:Bc:Gc:Rc. In the special observation mode, the light source control unit 21 controls each of the LEDs 20 a to 20 d so as to emit special light in which the light intensity ratio among the violet light V, the blue light B, the green light G, and the red light R is Vs:Bs:Gs:Rs. It is preferable that the special light can emphasize a structure such as a blood vessel at a specific depth. Further, as shown in FIG. 4, in the short wavelength observation mode, the light source control unit 21 controls each of the LEDs 20 a to 20 d so as to emit the violet light V which is the specific narrow-band light of the short wavelength.

In the present specification, the light intensity ratio includes a case where the ratio of at least one semiconductor light source is 0 (zero). Accordingly, a case where any one or two or more of the semiconductor light sources are not turned on is included. For example, as in the case where the light intensity ratio among the violet light V, the blue light B, the green light G, and the red light R is 1:0:0:0, even in a case where only one of the semiconductor light sources is turned on and the other three are not turned on, it is assumed that a light intensity ratio is present.

Further, the light source control unit 21 controls the light emission amount of the illumination light emitted from each of the LEDs 20 a to 20 d on the basis of brightness information sent from a brightness information calculation unit 54 of the processor device 16.

As shown in FIG. 2, the light guide 41 is built in the endoscope 12 and a universal cord (a cord that connects the endoscope 12 to the light source device 14 and the processor device 16), and propagates the light coupled by the optical path coupling unit 23 to the distal end portion 12 d of the endoscope 12. It is possible to use a multi-mode fiber as the light guide 41. As an example, it is possible to use a small-diameter fiber cable of which a core diameter is 105 μm, a cladding diameter is 125 μm, and a diameter including a protective layer as an outer skin is φ0.3 mm to φ0.5 mm.

An illumination optical system 30 a and an imaging optical system 30 b are provided in the distal end portion 12 d of the endoscope 12. The illumination optical system 30 a has the illumination lens 45, and the observation target is irradiated with the light from the light guide 41 through the illumination lens 45. The imaging optical system 30 b has an objective lens 46 and an image sensor 48. The reflected light from the observation target is incident on the image sensor 48 through the objective lens 46. In this manner, a reflected image of the observation target is formed on the image sensor 48.

The image sensor 48 is a color image sensor and captures a reflected image of the subject to output an image signal. The image sensor 48 is preferably a charge coupled device (CCD) image sensor, a complementary metal-oxide semiconductor (CMOS) image sensor, or the like. The image sensor 48 used in the invention is a color image sensor for obtaining RGBW image signals of four colors of red (R), green (G), blue (B), and white (W). That is, as shown in FIG. 5, the image sensor 48 comprises an R pixel (fourth pixel) provided with an R filter, a G pixel (second pixel) provided with a G filter, a B pixel (first pixel) provided with a B filter, and a W pixel (third pixel) provided with a W filter. The image sensor 48 is divided into two pixel groups of a first pixel group and a second pixel group, and the number of pixels of the second pixel group is greater than the number of pixels of the first pixel group. The first pixel group includes B pixels, and the second pixel group includes G pixels, R pixels, and W pixels.

In the image sensor 48, the W pixels are arranged in a checkered pattern. Further, for the signal obtained by the image sensor 48 in the short wavelength observation mode, in order to improve the resolution, as shown in demosaicing processing described later, by giving the G pixel a certain level or more of a sensitivity to the specific narrow-band light and using the correlation between the W pixel and the G pixel, a signal of the specific narrow-band light can be obtained even at a G pixel position. Similarly, by giving the R pixel a certain level or more of a sensitivity to the specific narrow-band light and using a correlation between the W pixel and the R pixel, a signal of the specific narrow-band light can be obtained even at an R pixel position.

As described above, in order to obtain a signal of specific narrow-band light at the G pixel position during illumination with specific narrow-band light of a short wavelength, as shown in FIG. 6, the transmittance of the G filter is set so that the G pixel has not only the sensitivity to light in a green band (first long-wavelength light) of 500 nm to 600 nm, but also a short-wavelength sensitivity to short-wavelength light (400 nm to 450 nm) including the specific narrow-band light of the short wavelength. In this manner, as shown in FIG. 7, in a case where the specific narrow-band light, the transmittance of the G filter, and the reflectance of each part (the reflectance of the “esophagus” is used in the case of FIG. 7, and the reflectances of the stomach and large intestine are shown by the dotted line) are multiplied, a received light intensity corresponding to the wavelength range of the specific narrow-band light is obtained as a received light intensity of the G pixel. A received light intensity of the B pixel is obtained by multiplying the specific narrow-band light, the transmittance of the B filter (blue band of 400 nm to 500 nm), and the reflectance of each part, and is a constant multiple of the intensity value of the received light intensity of the G pixel.

In a case where the signal value obtained by the G pixel is small, the influence of noise increases, and the image quality of the image after the demosaicing processing deteriorates. Therefore, for example, a short-wavelength sensitivity of the G pixel is preferably 10% or more of the maximum sensitivity of the G pixel at 400 nm to 450 nm. That is, in the G filter, the transmittance at 400 nm to 450 nm is preferably 10% or more of the transmittance at the wavelength range (for example, 540 nm to 560 nm) corresponding to the maximum sensitivity of the G pixel. Alternatively, the short-wavelength sensitivity of the G pixel is preferably 10% or more of a short-wavelength sensitivity of the B pixel. That is, in the G filter, the transmittance at 400 nm to 450 nm is preferably 10% or more of the transmittance at 400 nm to 450 nm of the B filter. The maximum sensitivity of the G pixel refers to the sensitivity to light in the wavelength range in which the transmittance of the G color filter is equal to or higher than a certain value (for example, 70%). The same applies to the maximum sensitivity of the R pixel.

On the other hand, in a case where the short-wavelength sensitivity of the G pixel becomes too high, the color reproducibility in a normal image deteriorates. Therefore, the short-wavelength sensitivity of the G pixel is preferably 35% or less of the maximum sensitivity of the G pixel at 400 nm to 450 nm. That is, in the G filter, the transmittance at 400 nm to 450 nm is preferably 35% or less of the transmittance at the wavelength range (for example, 540 nm to 560 nm) corresponding to the maximum sensitivity of the G pixel. Alternatively, the short-wavelength sensitivity of the G pixel is preferably 35% or less of the short-wavelength sensitivity of the B pixel. That is, in the G filter, the transmittance at 400 nm to 450 nm is preferably 35% or less of the transmittance at 400 nm to 450 nm of the B filter.

Further, in order to obtain a signal of specific narrow-band light at the R pixel position during illumination with specific narrow-band light of a short wavelength, as shown in FIG. 6, the transmittance of the R filter is set so that the R pixel has not only the sensitivity to light in a red band (second long-wavelength light) of 600 nm to 700 nm, but also the short-wavelength sensitivity to short-wavelength light (a wavelength band is 450 nm or less, for example, 400 nm to 450 nm) including the specific narrow-band light of the short wavelength. In this manner, as shown in FIG. 8, in a case where the specific narrow-band light, the transmittance of the R filter, and the reflectance of each part (the reflectance of the “esophagus” is used in the case of FIG. 8, and the reflectances of the stomach and large intestine are shown by the dotted line) are multiplied, a received light intensity corresponding to the wavelength range of the specific narrow-band light is obtained as a received light intensity of the R pixel. The received light intensity of the B pixel is a constant multiple of the received light intensity of the G pixel, similarly to the above.

Here, in a case where the signal value obtained by the R pixel is small, the influence of noise increases, and the image quality of the image after the demosaicing processing deteriorates. Therefore, for example, a short-wavelength sensitivity of the R pixel is preferably 10% or more of the maximum sensitivity of the R pixel at 400 nm to 450 nm. That is, in the R filter, the transmittance at 400 nm to 450 nm is preferably 10% or more of the transmittance at the wavelength range (for example, 640 nm to 660 nm) corresponding to the maximum sensitivity of the R pixel. Alternatively, the short-wavelength sensitivity of the R pixel is preferably 10% or more of the short-wavelength sensitivity of the B pixel. That is, in the R filter, the transmittance at 400 nm to 450 nm is preferably 10% or more of the transmittance at 400 nm to 450 nm of the B filter.

On the other hand, in a case where the short-wavelength sensitivity of the R pixel becomes too high, the color reproducibility in a normal image deteriorates. Therefore, the short-wavelength sensitivity of the R pixel is preferably 35% or less of the maximum sensitivity of the R pixel at 400 nm to 450 nm. That is, in the R filter, the transmittance at 400 nm to 450 nm is preferably 35% or less of the transmittance at the wavelength range (for example, 640 nm to 660 nm) corresponding to the maximum sensitivity of the R pixel. Alternatively, the short-wavelength sensitivity of the R pixel is preferably 35% or less of the short-wavelength sensitivity of the B pixel. That is, in the R filter, the transmittance at 400 nm to 450 nm is preferably 35% or less of the transmittance at 400 nm to 450 nm of the B filter.

In the image sensor 48, since the W pixel has a spectral sensitivity to the illumination light of the broadband light from the blue band to the red band so that the W pixel includes the specific narrow-band light of the short wavelength, the light in the green band, and the light in the red band, the saturation of the pixel value is faster than other pixels such as the B pixel, the G pixel, and the R pixel. Accordingly, the sensitivity of the W pixel is set lower than the sensitivity of other pixels such as the B pixel, the G pixel, and the R pixel. Specifically, the sensitivity of the W pixel is preferably 30% to 50% with respect to the maximum sensitivity of the B pixel (for example, the sensitivity of 440 nm to 460 nm) or the maximum sensitivity of the G pixel (for example, the sensitivity of 540 nm to 560 nm).

For example, in the relationship regarding the sensitivity between the W pixel and the G pixel, as shown in FIG. 9, the transmittance of the W filter is set so that a received light intensity of the W pixel obtained by multiplying the light source spectrum of the normal light and the transmittance of the W filter is about 50% (for example, 45% to 55%) of the received light intensity of the G pixel obtained by multiplying the light source spectrum of normal light and the transmittance of the G filter. Alternatively, in the relationship regarding the sensitivity between the W pixel and the B pixel, as shown in FIG. 10, the transmittance of the W filter is set so that the received light intensity of the W pixel obtained by multiplying the light source spectrum of the normal light and the transmittance of the W filter is about 50% (for example, 45% to 55%) of the received light intensity of the B pixel obtained by multiplying the light source spectrum of normal light, the transmittance of the B filter, and the reflectance of the esophagus.

The image sensor 48 may be a so-called complementary color image sensor comprising complementary color filters of cyan (C), magenta (M), yellow (Y), and green (G) instead of the RGBW color image sensor. In a case of using the complementary color image sensor, since four color image signals of CMYG are output, it is necessary to convert the four color image signals of CMYG into three color image signals of RGB by complementary color-primary color conversion. The image sensor 48 may be a monochrome image sensor without a color filter.

As shown in FIG. 2, the image signal output from the image sensor 48 is transmitted to a CDS/AGC circuit 50. The CDS/AGC circuit 50 performs correlated double sampling (CDS) and auto gain control (AGC) on an image signal which is an analog signal. The image signal that has passed through the CDS/AGC circuit 50 is converted into a digital image signal by an analog/digital (A/D) converter 52. The A/D converted digital image signal is input to the processor device 16.

The processor device 16 comprises an image acquisition unit 53, the brightness information calculation unit 54, a digital signal processor (DSP) 56, a noise removal unit 58, an image processing unit 60, a parameter switching unit 62, and a display control unit 64.

The image acquisition unit 53 acquires an observation image obtained by imaging the observation target with the endoscope 12. Specifically, a digital color image signal from the endoscope 12 is input to the image acquisition unit 53 as an observation image. The color image signal is an RGBW image signal composed of an R image signal output from the R pixel of the image sensor 48, a G image signal output from the G pixel of the image sensor 48, a B image signal output from the B pixel of the image sensor 48, and a W image signal output from the W pixel of the image sensor 48. The brightness information calculation unit 54 calculates brightness information indicating the brightness of the observation target on the basis of the RGBW image signal input from the image acquisition unit 53. The calculated brightness information is sent to the light source control unit 21 and is used to control the light emission amount of the illumination light.

The DSP 56 performs various kinds of signal processing, such as defect correction processing, offset processing, gain correction processing, linear matrix processing, or gamma conversion processing, on the received image signal. In the defect correction processing, a signal of a defective pixel of the image sensor 48 is corrected. In the offset processing, a dark current component is removed from the image signal subjected to the defect correction processing, and an accurate zero level is set. In the gain correction processing, a signal level is adjusted by multiplying the image signal after the offset processing by a specific gain. The linear matrix processing for improving the color reproducibility is performed on the image signal after the gain correction processing. After that, brightness and saturation are adjusted by the gamma conversion processing.

The demosaicing processing (also referred to as isotropic processing or synchronization processing) is performed on the image signal after the linear matrix processing in the demosaicing processing unit 56 a. In the normal observation mode and the special observation mode, a signal of the color lacking in each pixel is generated by interpolation. By the demosaicing processing, all pixels have signals of RGB colors. For the demosaicing processing in a case where a color image sensor for obtaining an RGBW image signal is used, for example, the method disclosed in JP2011-055038A can be used.

In the short wavelength observation mode, the observation target is illuminated with only the specific narrow-band light of the short wavelength. However, regarding the spectral sensitivity of a normal RGB image sensor (an image sensor consisting of R pixels, G pixels, and B pixels), as shown in FIG. 15, the G filter and the R filter have almost no transmittance in the specific narrow-band, and the G pixel and the R pixel have almost no sensitivity to the specific narrow-band light. Therefore, in the short wavelength observation image obtained on the basis of the specific narrow-band light of the short wavelength, signals are obtained only from the B pixel and the W pixel having a sensitivity to specific narrow-band light, and almost no signals are obtained from the G pixel and the R pixel. For this reason, even if the demosaicing processing based on the correlation with the W pixel is performed, a short wavelength observation image with high resolution cannot be obtained. However, as shown in the present embodiment, by giving the G pixel and the R pixel the sensitivity to the specific narrow-band light, a constant signal corresponding to the specific narrow-band light can be obtained also from the G pixel and the R pixel, and therefore, a short wavelength observation image with high resolution can be obtained by the demosaicing processing based on the correlation with the W pixel.

Hereinafter, in the short wavelength observation mode, by the signals obtained from the B pixel and the W pixel and the demosaicing processing based on the correlation with the W pixel, the signals obtained from the G pixel and the R pixel have a constant signal corresponding to the specific narrow-band light, and therefore, each signal is referred to as a short wavelength observation signal.

In the demosaicing processing in the short wavelength observation mode, based on the correlation with the pixel value of the W pixel, the pixel value of the B pixel at the G pixel position is calculated as a short wavelength observation signal, and the pixel value of the B pixel at the R pixel position is calculated as a short wavelength observation signal. In this manner, the G pixel position and the R pixel position also have the signal value corresponding to the specific narrow-band light, so that the resolution of the short wavelength observation image can be improved.

The method of calculating the pixel value of B pixel at the G pixel position as the short wavelength observation signal by the demosaicing processing based on the correlation with the pixel value of the W pixel is as follows. For example, as shown in FIG. 11, in an image signal 80 output from the image sensor 48, in a case of calculating a pixel value Bd of the B pixel at the G pixel position at a specific position SP, for discrete W pixel signals included in the image signal 80, W pixel values at all pixel positions are calculated by direction discrimination interpolation according to the neighboring W pixel values, image blur reduction processing for reducing image blur is performed on the calculated W pixel values by Wiener filter processing, and a W pixel signal (=Wd) after the image blur reduction processing (an image blur reduction processed signal) is calculated. Next, as shown in FIG. 12, by multiplying the pixel values of a 5^(x) 5 pixel area (a specific pixel area) by a first smoothing filter 82, a low frequency component mW (a first smooth-filtered component) is obtained. The first smoothing filter 82 is provided with a filter coefficient (such as “1” or “2”) at the position where the W pixel is present in the image signal 80.

Next, as shown in FIG. 13, by multiplying the pixel values of the 5×5 pixel area by a second smoothing filter 84, a low frequency component mBx (a second smooth-filtered component) is obtained. The second smoothing filter 84 is a filter that is applied to each 5×5 pixel area in which the G pixel is located at the specific position SP, and is provided with a filter coefficient (such as “1”) at the position where the B pixel is present in the image signal 80. In the second smoothing filter 84, the filter coefficient setting is performed so that the pixel values of the B pixels at positions close to the specific position SP are evenly acquired. Then, the pixel value Bd of the B pixel at the G pixel position is calculated by the following Equation X).

Bd=(mBx/mW)×Wd  Equation X)

The method of calculating the pixel value of B pixel at the R pixel position as the short wavelength observation signal by the demosaicing processing based on the correlation with the pixel value of the W pixel is as follows. Similar to the above case, a blur correction white (W) signal (=Wd) and the low frequency component mW are acquired. Then, as shown in FIG. 14, by multiplying the pixel values of the 5×5 pixel area by a third smoothing filter 86, a low frequency component mBy (a third smoothing filtered component) is acquired. The third smoothing filter 86 is a filter that is applied to each 5×5 pixel area in which the R pixel is located at the specific position SP, and is provided with a filter coefficient (such as “1”) at the position where the B pixel is present in the image signal 80. Then, the pixel value Bd of the B pixel at the R pixel position is calculated by the following Equation Y).

Bd=(mBy/mW)×Wd  Equation Y)

The noise removal unit 58 removes noise from the RGB image signal by performing noise removal processing (for example, a moving average method or a median filter method) on the image signal that has been subjected to gamma correction or the like by the DSP 56. The image signal from which the noise has been removed is transmitted to the image processing unit 60.

The image processing unit 60 performs image processing corresponding to each observation mode. As an image processing method, for example, there is a method in which image processing parameters such as gradation processing and saturation enhancement corresponding to each observation mode are prepared, and the image signal is multiplied by the image processing parameters according to each observation mode. In the case of the normal observation mode, the RGB image signal is multiplied by a parameter for the normal observation mode, and in the case of the special observation mode, the RGB image signal is multiplied by a parameter for the special observation mode. In the case of the short wavelength observation mode, the B image signal is multiplied by a parameter for the short wavelength observation mode. The parameter for the normal observation mode, the parameter for the special observation mode, and the parameter for the short wavelength observation mode described above are switched by the parameter switching unit 62 in accordance with the mode switching of the mode switching SW 13 a.

The display control unit 64 performs control for displaying an image signal input from the image processing unit 60 as an image that can be displayed on the monitor 18. In the case of the normal observation mode, a normal image is displayed on the monitor 18 by assigning the R image signal to the R channel of the monitor 18, the G image signal to the G channel of the monitor 18, and the B image signal to the B channel of the monitor 18. In the case of the special observation mode, a special observation image is displayed on the monitor 18 by assigning the G image signal to the R channel of the monitor 18, and the B image signal to the G channel and the B channel of the monitor 18 (in assigning, it is preferable to perform gradation processing and saturation enhancement). On the other hand, in the case of the short wavelength observation mode, a short wavelength observation image is displayed on the monitor 18 by assigning the short wavelength observation signal to each of the R, G, and B channels of the monitor 18. In assigning the short wavelength observation signal, the short wavelength observation image is multiplied by a gain for each of the R, G, and B channels, and the like, and then assigned to each channel. In this manner, the short wavelength observation image showing a structure that can be observed with specific narrow-band light of a short wavelength can be displayed on the monitor 18 as an image having a specific background color in which the structure of blood vessels and the like has better visibility than a gray image.

In the above embodiment, in the demosaicing processing in the short wavelength observation mode, the pixel value (B image signal) of the B pixel at the G pixel position and the R pixel position is calculated; however, similarly to the normal observation mode or the special observation mode, the pixel value of each of the R pixel, G pixel, and B pixel at each pixel position may be calculated.

In the embodiment, the hardware structure of the processing units included in the processor device 16, such as the image acquisition unit 53, the brightness information calculation unit 54, the DSP 56, the noise removal unit 58, the image processing unit 60, the parameter switching unit 62, and the display control unit 64 is various processors as follows. The various processors include a central processing unit (CPU) that is a general-purpose processor that functions as various processing units by executing software (program), a programmable logic device (PLD) that is a processor whose circuit configuration can be changed after manufacture, such as field programmable gate array (FPGA), a dedicated electrical circuit that is a processor having a circuit configuration designed exclusively for executing various types of processing, and the like.

One processing unit may be configured by one of various processors, or may be configured by a combination of two or more processors of the same type or different types (for example, a combination of a plurality of FPGAs or a combination of a CPU and an FPGA). In addition, a plurality of processing units may be configured by one processor. As an example of configuring a plurality of processing units by one processor, first, as represented by a computer, such as a client or a server, there is a form in which one processor is configured by a combination of one or more CPUs and software and this processor functions as a plurality of processing units. Second, as represented by a system on chip (SoC) or the like, there is a form of using a processor for realizing the function of the entire system including a plurality of processing units with one integrated circuit (IC) chip. Thus, various processing units are configured by using one or more of the above-described various processors as hardware structures.

More specifically, the hardware structure of these various processors is an electrical circuit (circuitry) in the form of a combination of circuit elements, such as semiconductor elements.

In the embodiment, the invention is applied to the endoscope system that performs processing on the endoscopic image as one of the medical images. However, the invention can also be applied to a medical image processing system that processes medical images other than the endoscopic image.

EXPLANATION OF REFERENCES

-   -   10: endoscope system     -   12: endoscope     -   12 a: insertion part     -   12 b: operation part     -   12 c: bendable portion     -   12 d: distal end portion     -   12 e: angle knob     -   13 a: mode switching SW     -   13 b: still image acquisition instruction part     -   14: light source device     -   16: processor device     -   18: monitor     -   19: user interface     -   20: light source unit     -   20 a: violet light emitting diode (V-LED)     -   20 b: blue light emitting diode (B-LED)     -   20 c: green light emitting diode (G-LED)     -   20 d: red light emitting diode (R-LED)     -   21: light source control unit     -   23: optical path coupling unit     -   30 a: illumination optical system     -   30 b: imaging optical system     -   41: light guide     -   45: illumination lens     -   46: objective lens     -   48: image sensor     -   50: CDS/AGC circuit     -   52: A/D converter     -   53: image acquisition unit     -   54: brightness information calculation unit     -   56: digital signal processor (DSP)     -   56 a: demosaicing processing unit     -   58: noise removal unit     -   60: image processing unit     -   62: parameter switching unit     -   64: display control unit     -   80: image signal     -   82: first smoothing filter     -   84: second smoothing filter     -   86: third smoothing filter 

What is claimed is:
 1. A medical image processing system comprising: a light source unit that emits specific narrow-band light of a short wavelength; and an image sensor that images an observation target illuminated with the specific narrow-band light, the image sensor including a first pixel group including a first pixel and a second pixel group including at least a second pixel and a third pixel, wherein the first pixel has a higher sensitivity to the specific narrow-band light than the second pixel, the second pixel has a sensitivity to first long-wavelength light of a longer wavelength than the specific narrow-band light and the specific narrow-band light, and the third pixel has a sensitivity to broadband illumination light including the specific narrow-band light and the first long-wavelength light.
 2. The medical image processing system according to claim 1, wherein, in the sensitivity of the second pixel, a short-wavelength sensitivity to short-wavelength light including the specific narrow-band light is 10% or more of a maximum sensitivity of the second pixel, or 10% or more of the short-wavelength sensitivity of the first pixel.
 3. The medical image processing system according to claim 1, wherein, in the sensitivity of the second pixel, a short-wavelength sensitivity to short-wavelength light including the specific narrow-band light is 35% or less of a maximum sensitivity of the second pixel, or 35% or less of the short-wavelength sensitivity of the first pixel.
 4. The medical image processing system according to claim 2, wherein, in the sensitivity of the second pixel, a short-wavelength sensitivity to short-wavelength light including the specific narrow-band light is 35% or less of a maximum sensitivity of the second pixel, or 35% or less of the short-wavelength sensitivity of the first pixel.
 5. The medical image processing system according to claim 1, wherein the sensitivity of the third pixel is 30% or more and 50% or less with respect to a maximum sensitivity of the first pixel or the second pixel.
 6. The medical image processing system according to claim 2, wherein the sensitivity of the third pixel is 30% or more and 50% or less with respect to a maximum sensitivity of the first pixel or the second pixel.
 7. The medical image processing system according to claim 3, wherein the sensitivity of the third pixel is 30% or more and 50% or less with respect to a maximum sensitivity of the first pixel or the second pixel.
 8. The medical image processing system according to claim 1, wherein a pixel value of the first pixel at a pixel position of the second pixel is calculated by demosaicing processing based on a correlation with a pixel value of the third pixel.
 9. The medical image processing system according to claim 2, wherein a pixel value of the first pixel at a pixel position of the second pixel is calculated by demosaicing processing based on a correlation with a pixel value of the third pixel.
 10. The medical image processing system according to claim 3, wherein a pixel value of the first pixel at a pixel position of the second pixel is calculated by demosaicing processing based on a correlation with a pixel value of the third pixel.
 11. The medical image processing system according to claim 3, wherein a pixel value of the first pixel at a pixel position of the second pixel is calculated by demosaicing processing based on a correlation with a pixel value of the third pixel.
 12. The medical image processing system according to claim 8, wherein the demosaicing processing includes: acquiring an image blur reduction processed signal in which image blur reduction processing is performed on an image signal output from the image sensor; acquiring a first smooth-filtered component by multiplying the image signal by a first smoothing filter having a filter coefficient at a position corresponding to the third pixel; and acquiring a second smooth-filtered component by multiplying the image signal by a second smoothing filter that is applied to each specific pixel area in which the second pixel is located at a specific position, the second smoothing filter having a filter coefficient at a position corresponding to the first pixel in the specific pixel area, and the pixel value of the first pixel at the pixel position of the second pixel is calculated on the basis of the image blur reduction processed signal, the first smooth-filtered component, and the second smooth-filtered component.
 13. The medical image processing system according to claim 1, wherein the second pixel group includes a fourth pixel, and the fourth pixel has a sensitivity to second long-wavelength light of a longer wavelength than the first long-wavelength light and the specific narrow-band light.
 14. The medical image processing system according to claim 2, wherein the second pixel group includes a fourth pixel, and the fourth pixel has a sensitivity to second long-wavelength light of a longer wavelength than the first long-wavelength light and the specific narrow-band light.
 15. The medical image processing system according to claim 13, wherein, in the sensitivity of the fourth pixel, a short-wavelength sensitivity to short-wavelength light including the specific narrow-band light is 10% or more of a maximum sensitivity of the fourth pixel, or 10% or more of the short-wavelength sensitivity of the first pixel.
 16. The medical image processing system according to claim 13, wherein, in the sensitivity of the fourth pixel, a short-wavelength sensitivity to short-wavelength light including the specific narrow-band light is 35% or less of a maximum sensitivity of the fourth pixel, or 35% or less of the short-wavelength sensitivity of the first pixel.
 17. The medical image processing system according to claim 7, wherein a pixel value of the first pixel at a pixel position of the fourth pixel is calculated by demosaicing processing based on a correlation with a pixel value of the third pixel.
 18. The medical image processing system according to claim 17, wherein the demosaicing processing includes: acquiring an image blur reduction processed signal in which image blur reduction processing is performed on an image signal output from the image sensor; acquiring a first smooth-filtered component by multiplying the image signal by a first smoothing filter having a filter coefficient at a position corresponding to the third pixel; and acquiring a third smooth-filtered component by multiplying the image signal by a third smoothing filter that is applied to a specific pixel area in which the fourth pixel is located at a specific position, the third smoothing filter having a filter coefficient at a position corresponding to the first pixel in the specific pixel area, and the pixel value of the first pixel at the pixel position of the fourth pixel is calculated on the basis of the image blur reduction processed signal, the first smooth-filtered component, and the third smooth-filtered component.
 19. The medical image processing system according to claim 1, wherein the number of pixels of the second pixel group is greater than the number of pixels of the first pixel group.
 20. The medical image processing system according to claim 1, wherein a center wavelength of the specific narrow-band light is included in a range of 400 nm to 450 nm, and a half-width of the specific narrow-band light is 40 nm or less. 